多模式变压器的最新努力通过合并视觉和文本信息改善了视觉上丰富的文档理解（VRDU）任务。但是，现有的方法主要集中于诸如单词和文档图像贴片之类的细粒元素，这使得他们很难从粗粒元素中学习，包括短语和显着视觉区域（如突出的图像区域）等自然词汇单元。在本文中，我们对包含高密度信息和一致语义的粗粒元素更为重要，这对于文档理解很有价值。首先，提出了文档图来模拟多层次多模式元素之间的复杂关系，其中通过基于群集的方法检测到显着的视觉区域。然后，提出了一种称为mmlayout的多模式变压器，以将粗粒的信息纳入基于图形的现有预训练的细颗粒的多峰变压器中。在mmlayout中，粗粒信息是从细粒度聚集的，然后在进一步处理后，将其融合到细粒度中以进行最终预测。此外，引入常识增强以利用天然词汇单元的语义信息。关于四个任务的实验结果，包括信息提取和文档问答，表明我们的方法可以根据细粒元素改善多模式变压器的性能，并使用更少的参数实现更好的性能。定性分析表明，我们的方法可以在粗粒元素中捕获一致的语义。

将深度学习与象征性逻辑推理相结合旨在利用这两个领域的成功，并引起越来越多的关注。受到深度循环的启发，这是一种端到端的模型，该模型训练了逻辑程序的推理，我们引入了Ima-Glove-GA，这是一种以自然语言表达的多步推理的迭代神经推理网络。在我们的模型中，推理是使用基于带门注意机制的RNN的迭代记忆神经网络进行的。我们在三个数据集上评估了iMa-glove-ga：副本，Conceptrules V1和Conceptrules V2。实验结果表明，与DeepLo​​gic和其他RNN基线模型相比，深沟和栅极注意可以达到更高的测试精度。当规则被淘汰时，我们的模型比罗伯塔·洛尔格（Roberta-Large）实现了更好的分布概括。此外，为了解决当前多步推理数据集中推理深度分布不平衡分布的问题，我们开发了Pararule-Plus，这是一个大型数据集，其中包含更多需要更深入推理步骤的示例。实验结果表明，添加Pararule-Plus可以在需要更深层次深度的示例中提高模型的性能。源代码和数据可在https://github.com/strong-ai-lab/multi-step-deductive-reasoning-over-natural语言中获得。

在像地震等自然灾害后建立损伤检测对于启动有效的应急行动至关重要。远程感测的非常高空间分辨率（VHR）图像可以提供由于它们具有高几何精度的受影响建筑物的能力而提供重要信息。已经开发出许多方法来检测由于地震因地震而受损的建筑物。但是，使用深神经网络（DNN）已经支付了利用VHR图像中所代表的丰富的功能。本文提出了一种基于DNN和改进的分段方法的新型超像素的方法，从VHR图像中检测损坏的建筑物。首先，扩展了修改的快速扫描和自适应合并方法以创建初始过分分割。其次，基于相邻图（RAG）的区域合并段，被认为是由局部二进制模式（LBP）纹理，光谱和形状特征组成的改进的语义相似性标准。第三，呈现了使用堆叠的去噪自动编码器的预训练的DNN，称为SDAE-DNN，以利用丰富的语义特征来构建损坏检测。 SDAE-DNN的深层特征抽象可以通过学习更多内在和鉴别特征来提高检测精度，这使得使用最先进的替代分类器的其他方法表现优于其他方法。我们展示了我们在尼泊尔Bhaktapur的复杂城市地区使用WorldView-2图像的方法的可行性和有效性，这是受2015年4月25日的尼泊尔地震影响的。

In recent years, arbitrary image style transfer has attracted more and more attention. Given a pair of content and style images, a stylized one is hoped that retains the content from the former while catching style patterns from the latter. However, it is difficult to simultaneously keep well the trade-off between the content details and the style features. To stylize the image with sufficient style patterns, the content details may be damaged and sometimes the objects of images can not be distinguished clearly. For this reason, we present a new transformer-based method named STT for image style transfer and an edge loss which can enhance the content details apparently to avoid generating blurred results for excessive rendering on style features. Qualitative and quantitative experiments demonstrate that STT achieves comparable performance to state-of-the-art image style transfer methods while alleviating the content leak problem.

In recent years, the Transformer architecture has shown its superiority in the video-based person re-identification task. Inspired by video representation learning, these methods mainly focus on designing modules to extract informative spatial and temporal features. However, they are still limited in extracting local attributes and global identity information, which are critical for the person re-identification task. In this paper, we propose a novel Multi-Stage Spatial-Temporal Aggregation Transformer (MSTAT) with two novel designed proxy embedding modules to address the above issue. Specifically, MSTAT consists of three stages to encode the attribute-associated, the identity-associated, and the attribute-identity-associated information from the video clips, respectively, achieving the holistic perception of the input person. We combine the outputs of all the stages for the final identification. In practice, to save the computational cost, the Spatial-Temporal Aggregation (STA) modules are first adopted in each stage to conduct the self-attention operations along the spatial and temporal dimensions separately. We further introduce the Attribute-Aware and Identity-Aware Proxy embedding modules (AAP and IAP) to extract the informative and discriminative feature representations at different stages. All of them are realized by employing newly designed self-attention operations with specific meanings. Moreover, temporal patch shuffling is also introduced to further improve the robustness of the model. Extensive experimental results demonstrate the effectiveness of the proposed modules in extracting the informative and discriminative information from the videos, and illustrate the MSTAT can achieve state-of-the-art accuracies on various standard benchmarks.

Machine learning models are typically evaluated by computing similarity with reference annotations and trained by maximizing similarity with such. Especially in the bio-medical domain, annotations are subjective and suffer from low inter- and intra-rater reliability. Since annotations only reflect the annotation entity's interpretation of the real world, this can lead to sub-optimal predictions even though the model achieves high similarity scores. Here, the theoretical concept of Peak Ground Truth (PGT) is introduced. PGT marks the point beyond which an increase in similarity with the reference annotation stops translating to better Real World Model Performance (RWMP). Additionally, a quantitative technique to approximate PGT by computing inter- and intra-rater reliability is proposed. Finally, three categories of PGT-aware strategies to evaluate and improve model performance are reviewed.

We propose a novel approach to self-supervised learning of point cloud representations by differentiable neural rendering. Motivated by the fact that informative point cloud features should be able to encode rich geometry and appearance cues and render realistic images, we train a point-cloud encoder within a devised point-based neural renderer by comparing the rendered images with real images on massive RGB-D data. The learned point-cloud encoder can be easily integrated into various downstream tasks, including not only high-level tasks like 3D detection and segmentation, but low-level tasks like 3D reconstruction and image synthesis. Extensive experiments on various tasks demonstrate the superiority of our approach compared to existing pre-training methods.

Collaboration among industrial Internet of Things (IoT) devices and edge networks is essential to support computation-intensive deep neural network (DNN) inference services which require low delay and high accuracy. Sampling rate adaption which dynamically configures the sampling rates of industrial IoT devices according to network conditions, is the key in minimizing the service delay. In this paper, we investigate the collaborative DNN inference problem in industrial IoT networks. To capture the channel variation and task arrival randomness, we formulate the problem as a constrained Markov decision process (CMDP). Specifically, sampling rate adaption, inference task offloading and edge computing resource allocation are jointly considered to minimize the average service delay while guaranteeing the long-term accuracy requirements of different inference services. Since CMDP cannot be directly solved by general reinforcement learning (RL) algorithms due to the intractable long-term constraints, we first transform the CMDP into an MDP by leveraging the Lyapunov optimization technique. Then, a deep RL-based algorithm is proposed to solve the MDP. To expedite the training process, an optimization subroutine is embedded in the proposed algorithm to directly obtain the optimal edge computing resource allocation. Extensive simulation results are provided to demonstrate that the proposed RL-based algorithm can significantly reduce the average service delay while preserving long-term inference accuracy with a high probability.

The traditional statistical inference is static, in the sense that the estimate of the quantity of interest does not affect the future evolution of the quantity. In some sequential estimation problems however, the future values of the quantity to be estimated depend on the estimate of its current value. This type of estimation problems has been formulated as the dynamic inference problem. In this work, we formulate the Bayesian learning problem for dynamic inference, where the unknown quantity-generation model is assumed to be randomly drawn according to a random model parameter. We derive the optimal Bayesian learning rules, both offline and online, to minimize the inference loss. Moreover, learning for dynamic inference can serve as a meta problem, such that all familiar machine learning problems, including supervised learning, imitation learning and reinforcement learning, can be cast as its special cases or variants. Gaining a good understanding of this unifying meta problem thus sheds light on a broad spectrum of machine learning problems as well.

Most Graph Neural Networks follow the message-passing paradigm, assuming the observed structure depicts the ground-truth node relationships. However, this fundamental assumption cannot always be satisfied, as real-world graphs are always incomplete, noisy, or redundant. How to reveal the inherent graph structure in a unified way remains under-explored. We proposed PRI-GSL, a Graph Structure Learning framework guided by the Principle of Relevant Information, providing a simple and unified framework for identifying the self-organization and revealing the hidden structure. PRI-GSL learns a structure that contains the most relevant yet least redundant information quantified by von Neumann entropy and Quantum Jensen-Shannon divergence. PRI-GSL incorporates the evolution of quantum continuous walk with graph wavelets to encode node structural roles, showing in which way the nodes interplay and self-organize with the graph structure. Extensive experiments demonstrate the superior effectiveness and robustness of PRI-GSL.